Test reporting
HTML Test Report (preview)
The HTML Test Report integrates the artifacts generated during test execution and combines them with test documentation and meta information into an interactive webpage.
Generating the test report
Currently the test report can only be generated if the test cases have been executed using the runTests
gradle task (e.g. .\gradlew.bat runTests
).
To generate the test report run the createTestReport
task: .\gradlew.bat createTestReport
.
The test report will be generated into the MiniHilProject/build/tests/testReport
folder.
In addition to the HTML Report MiniHILTestReport.html
the reports content is also generated in XML and json format.
Machine readable report formats
In addition to the HTML report the test results are also generated in XML and json format. You can find the generated files as well as schema files for the XML (MiniHILTestReport.xsd
) and json (MiniHILTestReport.schema.json
) format in the MiniHilProject/build/tests/testReport
folder.
Both, the XML and json format, are based on the same data model and contain the same information.They may contain additional information which are not rendered in the HTML report.
Documenting test cases
In addition to the test results the test report may also contain test documentation. If a cage file contains a comment block for a test case the content of this block is included in the generated HTML report. To format the content Markdown may be used. For a comment block to be associated with the test case the comment block must be placed directly before the test case definition.
The following example shows a test case definition with a comment block. The comment block is associated with the test case TemplateTestCase1
and will be included in the HTML report.
TestSuite TemplateTestSuite:
TemplateTestCase1
;
/*
## This is a template for a test case
### Description
1. Description of the `test case`
2. Description of the *test italic case*
3. Description of the **test bold case**
[google.com link](http://www.google.com)
```
Some
multiline
code
ABCd
```
---
Some more text.
*/
Step TemplateTestCase1:
timeout 100 ms
action
timekeeper.startTimeMeasurement
wait 50 ms
action
timekeeper.stopTimeMeasurement
;
Adding test case meta information
For easier processing of the test reports by downstream tools such as ALM importers or test management tools the test report may contain meta information for each test case. The meta information is defined using the @MetaInfo
annotation. The annotation may be used multiple times to define multiple meta information entries for a test case.
Meta information are arbitrary key-value pairs which may be freely chosen by the user.
Here is an example of a test case definition with meta information followed by the resulting json output:
TestSuite TemplateTestSuite:
TemplateTestCase1
;
/**
My test case doc here...
*/
@MetaInfo(key="almId", text="TST-123")
@MetaInfo(key="state", text="WIP")
Step TemplateTestCase1:
timeout 100 ms
action
timekeeper.startTimeMeasurement
wait 50 ms
action
timekeeper.stopTimeMeasurement
;
{
//[....]
"testSuites" : [ {
"testCases" : [ {
"metaInfo" : {
"almId" : "TST-123",
"state" : "WIP"
},
"comment" : "My test case doc here...",
"stdout" : "\n[==============================================================================]\n[ BEGIN ] TestSuite TemplateTestSuite\n\n\n[------------------------------------------------------------------------------]\n[ RUN ] TestCase TemplateTestCase1\n\n[ OK ] TestCase TemplateTestCase1\n\n[------------------------------------------------------------------------------]\n[ PASSED ] TestSuite TemplateTestSuite\n[ ] Executed: 1 Tests\n[ ] Passed : 1 Tests\n[ ] Skipped : 0 Tests\n[ ] Failed : 0 Tests\n\n[############################ Test Execution Finished ############################\nTest execution result: successful\n\n TS No: 0 [ PASSED] : TemplateTestSuite\n\n Total number of Test Cases: 1\n PASSED: 1\n SKIPPED: 0\n FAILED: 0\n",
"name" : "TemplateTestCase1",
"id" : 1,
"testSuite" : "TemplateTestSuite",
"result" : "passed",
"executionTime" : "0.056"
} ],
"testTraceDiagram" : "[.....]",
"name" : "TemplateTestSuite"
} ]
}